System and method for projected tool trajectories for surgical navigation systems

ABSTRACT

The present disclosure teaches a system and method for communicating the spatial position and orientation of surgical instruments with respect to a surgical area of interest. Using a visual display of a surgical site generated by a camera feed, a computer generates a virtual overlay of the location and projected trajectory of a surgical instrument based on its current position and orientation. Position and orientation information is generated and stored using tracking markers and a tracking sensor in information communication with the computer.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to International Application No.CA/2014/050266, titled “SYSTEM AND METHOD FOR DYNAMIC VALIDATION,CORRECTION OF REGISTRATION FOR SURGICAL NAVIGATION” and filed on Mar.14, 2014, the entire contents of which are incorporated herein byreference.

FIELD

The present disclosure relates to an apparatus and method forcommunicating to a user the relative distance of an object locatedwithin a surgical area of interest relative to a surgical instrumentoperating in said surgical area of interest. The present system may beused with any compatible surgical navigation system. A non-limitingexample of such a surgical navigation system is outlined in the PCTInternational application CA/2014/050270 entitled “SYSTEMS AND METHODSFOR NAVIGATION AND SIMULATION OF INVASIVE THERAPY”, which claims thepriority benefit of U.S. Provisional Patent Application Ser. Nos.61/800,155 and 61/924,993, and wherein for the purposes of this presentUnited States Patent Application, the Detailed Description, and Figuresof PCT International application CA2014050270 are incorporated herein byreference.

BACKGROUND

In presently performed minimally invasive navigated surgeries surgeonsoften operate on the patient through a small corridor such as an accessport. The corridors normally have very small openings for tools or othermedical equipment. This limits their visibility of the surgicaloperating area due to the small corridors and areas the operations takeplace in. To enhance visibility of the area they generally use a headsup display or microscope which shows the surgical site of interest at agreater magnification. But this results in issues with tool navigation,specifically depth perception, as with a single camera, depth of toolscannot be gauged by the surgeon.

Thus, there is a need for mechanisms to provide this information to thesurgeon in a consistent manner and one in which they can utilize withouthindering other aspects of the surgical procedure. The inventiondisclosed herein attempts to improve the depth perception of the surgeonby providing a mechanism for attaining and communicating suchinformation to the surgeon, thereby attempting to improve presentlyperformed minimally invasive surgeries.

SUMMARY

The present disclosure is generally related to image guided medicalprocedures using an access port. This port-based surgery approach allowsa surgeon, or robotic surgical system, to perform a surgical procedureinvolving tumor resection in which the residual tumor remaining after isminimized, while also minimizing the trauma to the intact white and greymatter of the brain. In such procedures, trauma may occur, for example,due to contact with the access port, stress to the brain matter,unintentional impact with surgical devices, and/or accidental resectionof healthy tissue.

Disclosed herein is a system and method for communicating a distance ofa surgical instrument from an object in a surgical area of interest. Anembodiment of a system for communicating a distance of a surgicalinstrument from an object in a surgical area of interest, comprises:

a surgical instrument;

at least one non-contact distance acquiring device in a known relativeposition and orientation with respect to the surgical instrument;

a computer processor, in data communication with the at least onenon-contact distance acquiring device, the computer processor beingprogrammed with instructions to compute a distance between said surgicalinstrument and the object in the surgical area of interest; and

a communication device for communicating the distance to a user.

A method for communicating a distance of a surgical instrument from anobject in a surgical area of interest, comprises:

determining a relative position and orientation between at least onenon-contact distance acquiring device and a surgical instrument;

acquiring a first distance, between said at least one non-contactdistance acquiring device and the object in the surgical area ofinterest;

computing, using the determined relative position and orientation andthe first distance, a second distance between the surgical instrumentand the object; and

communicating the second distance to a user.

Additionally, a camera for acquiring an image feed of the surgical areaof interest may be included. The camera having a known position andorientation with respect to the surgical instrument, and being ininformation communication with the computer processor. Said processorbeing programmed with instructions to overlay onto the image feed,generated on a visual display, a visual cue depicting the distancebetween said surgical instrument and the object. The overlay may alsodepict a projected trajectory of the surgical instrument. This projectedtrajectory may take the form of a line. The visual cue may inform a userof the distance from the surgical instrument to the object by changingthe characteristics of the line generated on the visual display at thepoint where the trajectory would intersect with the object.

A tracking system may be employed to determine the relative positionsand orientations of surgical equipment in the operating room such as oneor more of the camera, the surgical instrument and the non-contactdistance acquiring device. Using one or more tracking marker assembliesattachable to components of the surgical equipment, a tracking sensormay continuously monitor their relative positions and orientations.

The object in the surgical area of interest may include tissue of apatient being operated an implant, or other objects that wouldpotentially be located in the surgical operating area. The distanceacquiring device may be a laser range finder, a structured lightdetection device for 3D imaging, an ultrasonic transducer, or any othernon-contact device capable of determining the distance of an objectrelative to itself.

The camera may be an MRI, a CT scanner, an X-ray scanner, a PET scanner,an ultrasonic scanner or a digital camera. The visual display can be adigital display, a heads-up display, a monitor, a navigation instrumentdisplay or a microscope display.

A further understanding of the functional and advantageous aspects ofthe present disclosure can be realized by reference to the followingdetailed description and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments will now be described, by way of example only, withreference to the drawings, in which:

FIG. 1A illustrates the insertion of an access port into a human brain,for providing access to internal brain tissue during a medicalprocedure.

FIG. 1B is a diagram illustrating components of an exemplary surgicalsystem used in port based surgery.

FIG. 2 illustrates a surgical instrument with attached tracking markers.

FIG. 3 illustrates a surgical instrument with attached tracking markersand the alignment of its corresponding virtual overlay.

FIG. 4 illustrates a diagram depicting the union of two coordinatespaces.

FIG. 5 illustrates a diagram depicting the alignment of a virtual andactual imaging feed.

FIG. 6A illustrates a medical instrument with attached laser rangefinder and its movement trajectory.

FIG. 6B illustrates a medical instrument with attached laser rangefinder detecting an imminent object.

FIG. 6C illustrates a medical instrument with attached laser rangefinder overlay and its projected trajectory.

FIG. 6D illustrates a medical instrument with attached laser rangefinder detecting an imminent object and its corresponding overlayshowing its projected trajectory.

FIG. 7A illustrates a diagram of a medical instrument with an imminentobject and detection of the object using structured light.

FIG. 7B illustrates a diagram of an overlay of a medical instrument withits projected trajectory on a camera imaging feed.

FIG. 8 illustrates an exemplary embodiment of a projected overlay usedduring a mock surgical procedure relative to a mock brain.

DETAILED DESCRIPTION

Various embodiments and aspects of the disclosure will be described withreference to details discussed below. The following description anddrawings are illustrative of the disclosure and are not to be construedas limiting the disclosure. Numerous specific details are described toprovide a thorough understanding of various embodiments of the presentdisclosure. However, in certain instances, well-known or conventionaldetails are not described in order to provide a concise discussion ofembodiments of the present disclosure.

As used herein, the terms, “comprises” and “comprising” are to beconstrued as being inclusive and open ended, and not exclusive.Specifically, when used in the specification and claims, the terms,“comprises” and “comprising” and variations thereof mean the specifiedfeatures, steps or components are included. These terms are not to beinterpreted to exclude the presence of other features, steps orcomponents.

As used herein, the term “exemplary” means “serving as an example,instance, or illustration,” and should not be construed as preferred oradvantageous over other configurations disclosed herein.

As used herein, the terms “about” and “approximately” are meant to covervariations that may exist in the upper and lower limits of the ranges ofvalues, such as variations in properties, parameters, and dimensions. Inone non-limiting example, the terms “about” and “approximately” meanplus or minus 10 percent or less.

Unless defined otherwise, all technical and scientific terms used hereinare intended to have the same meaning as commonly understood to one ofordinary skill in the art. Unless otherwise indicated, such as throughcontext, as used herein, the following terms are intended to have thefollowing meanings:

As used herein, the phrase “access port” refers to a cannula, conduit,sheath, port, tube, or other structure that is insertable into asubject, in order to provide access to internal tissue, organs, or otherbiological substances. In some embodiments, an access port may directlyexpose internal tissue, for example, via an opening or aperture at adistal end thereof, and/or via an opening or aperture at an intermediatelocation along a length thereof. In other embodiments, an access portmay provide indirect access, via one or more surfaces that aretransparent, or partially transparent, to one or more forms of energy orradiation, such as, but not limited to, electromagnetic waves andacoustic waves.

As used herein the phrase “intraoperative” refers to an action, process,method, event or step that occurs or is carried out during at least aportion of a medical procedure. Intraoperative, as defined herein, isnot limited to surgical procedures, and may refer to other types ofmedical procedures, such as diagnostic and therapeutic procedures.

Various apparatuses or processes will be described below to provideexamples of embodiments of the invention. No embodiment described belowlimits any claimed invention and any claimed invention may coverprocesses or apparatuses that differ from those described below. Theclaimed inventions are not limited to apparatuses or processes havingall of the features of any one apparatus or process described below orto features common to multiple or all of the apparatuses or processesdescribed below. It is possible that an apparatus or process describedbelow is not an embodiment of any claimed invention.

Furthermore, numerous specific details are set forth in order to providea thorough understanding of the embodiments described herein. However,it will be understood by those of ordinary skill in the art that theembodiments described herein may be practiced without these specificdetails. In other instances, well known methods, procedures andcomponents have not been described in detail so as not to obscure theembodiments described herein. Also, the description is not to beconsidered as limiting the scope of the embodiments described herein.

Furthermore, in the following passages, different aspects of theembodiments are defined in more detail. In particular, any featureindicated as being preferred or advantageous may be combined with atleast one other feature or features indicated as being preferred oradvantageous.

Embodiments of the present disclosure provide overlays of medicalequipment for assisting a surgeon in visualizing a surgical area orobject of interest such as a medical instrument, and methods of usethereof. Some embodiments of the present disclosure relate to minimallyinvasive medical procedures that are performed via an access port,whereby surgery, diagnostic imaging, therapy, or other medicalprocedures (e.g. minimally invasive medical procedures) are performedbased on access to internal tissue through the access port.

An example of an access port is an intracranial conduit which may beemployed in neurological procedures in order to provide access tointernal tissue pathologies, such as tumors. One example of anintracranial access port is the BrainPath™ surgical access port providedby NICO, which may be inserted into the brain via an obturator with anatraumatic tip. Such an access port may be employed during a surgicalprocedure, by inserting the access port, via the obturator that isreceived within the access port, through the white and gray of the brainto access a surgical site.

Minimally invasive brain surgery using access ports is a recentlyconceived method of performing surgery on brain tumors previouslyconsidered inoperable. One object of the present invention is to providea system and method to assist in minimally invasive brain surgery. Toaddress intracranial surgical concerns, specific products such as theNICO BrainPath™ port have been developed for port-based surgery.

FIG. 1A illustrates the insertion of an access port 100 into a humanbrain 10, for providing access to internal brain tissue during a medicalprocedure. In FIG. 1A, access port 100 is inserted into a human brain10, providing access to internal brain tissue. Surgical instruments(which includes any surgical equipment a surgeon may insert into braintissue including surgical tools such as scalpels, needles, biopsyprobes, suctioning devices, scissors to mention just a few) may then beinserted within the lumen of the access port 100 in order to performsurgical, diagnostic and/or therapeutic procedures, such as resectingtumors as necessary.

As seen in FIG. 1A, port 100 is comprised of a cylindrical assemblyformed of an outer sheath. Port 100 may accommodate an introducer (notshown) which is an internal cylinder that slidably engages the internalsurface of port 100. The introducer may have a distal end in the form ofa conical atraumatic tip to allow for insertion into the sulcal folds ofthe brain 10. Port 100 has a sufficient diameter to enable bimanualmanipulation of the surgical instrument(s) within its annular volumesuch as suctioning devices, scissors, scalpels, and cutting devices asexamples.

FIG. 1B is a diagram illustrating components of an exemplary surgicalsystem used in port based surgery. FIG. 1B shows a navigation system 107having an equipment tower 101, tracking system 113, display 111 (for agraphical user interface), an intelligent positioning system 175 andtracking markers 165 used to track surgical instruments or access port100. Tracking system 113 may also be considered an optical trackingdevice which tracks the tracking markers 165. The tracking system mayinclude a tracking camera.

As shown in FIG. 1B, surgeon 103 is resecting a tumor in the brain of apatient 106, through port 100. External scope 104, attached to automatedarm 102, is typically used by the surgeon to enhance visibility of thebrain at the distal end of the port 100. The external scope 104 may bezoomed-in or zoomed-out, and its output depicted on a visual display 111which may be overlaid with a virtual imaging feed of virtual medicalinstruments contained in the field of view of the external scope 104.The overlays may include the medical instruments projected trajectories,as will be discussed in more detail below, allowing for thevisualization of the instruments trajectories and their respectivedistances from imminent structures.

In an embodiment, an overlay of a surgical instrument visualization andpatient imaging information on a video image feed of the surgical fieldis provided during a procedure. An example surgical instrument is shownat 210 in FIG. 2, which includes a pointer segment 212 and landmarks 200(four (4) shown) which is used to verify registration and locatepreoperatively determined anatomical structures during navigatedsurgical procedures.

In FIG. 3 an actual surgical tool 210 with its associated landmarks 200and its associated virtual object representation comprised of virtuallandmarks 300, virtual pointer segment 310, and projected extension of avirtual pointer segment 320 are shown. In this exemplary illustration,the virtual representation of the pointer tool 210 has a projectedextension 320 out of the distal end of the tool positioned along thetools central axis shown as a dotted line. This extension depicts thetrajectory of the tools distal tip given its path is coaxial with thepointer segment 310 of the tool. The projected extension 320 in additionto providing trajectory information also provides a visual cue as to thedistance of the end of the tool 210 from an imminent structure (eithertissue or any other form of detectable matter). The visual cue may beprovided by changing the colour, thickness, pattern or any othercharacteristic of the projected extension to portray the point at whichthe projected extension penetrates an imminent structure.

The surgical instrument 210 may be tracked with one or more sensorswhich are in communication with one or more transceiver(s) of thetracking system that receives, records and/or processes the informationregarding the instrument(s) that the sensor(s) are detecting. Thesensors may track, among other things, the spatial position of theinstrument(s), including its angle and orientation (i.e. pose).Information regarding the distance of the distal end of the tool 210from an imminent structure may be determined using a structured lightscan of the region in which the distal end of the instrument is located,a laser range detector located on the tool 210, or another applicablemechanism not described here.

Persons skilled in the art will appreciate that being able to visualizea medical instrument, its trajectory path, and distance from imminentstructures when it is within the vicinity of a patient will aid in theimprovement of the accuracy of, and time required for, the procedure.

Active or passive fiduciary markers may be placed on the port 100 and/orimaging sensor 104, and/or any medical instruments 210 to determine thelocation of these objects using the tracking camera 113 and navigationsystem.

These markers (such as 200 shown in FIG. 2) may be reflective spheresconfigured to be seen by the stereo camera of the tracking system toprovide identifiable points for tracking. A tracked instrument trackedby the tracking system 113 is typically defined by a grouping of markerssuch as markers 200 of instrument 210, which identify a volume and anyprojected extensions thereof, and are used to determine the spatialposition and pose of the volume of the tracked instrument in threedimensions. Typically, in known exemplary tracking systems a minimum ofthree spheres are required on a tracked tool to define the instrument'sspatial position and orientation; however it is known in the art thatthe use of four markers is preferred. For example tool 210 shown in FIG.2 uses four (4) optical tracking markers 200.

Markers may be arranged statically on a target on the outside of thepatient's body or connected thereto. Tracking data of the markersacquired by the stereo camera are then logged and tracked by thetracking system. An advantageous feature is the selection of markersthat can be segmented easily by the tracking system against backgroundsignals. For example, infrared (IR)-reflecting markers and an IR lightsource from the direction of the stereo camera can be used. Such atracking system is known, for example, the “Polaris” system availablefrom Northern Digital Inc.

In an embodiment, the navigation system may utilize reflective sphericalmarkers in combination with a stereo camera system, to determine spatialpositioning and pose of the medical instruments and other objects withinthe operating theater. Differentiation of the types of medicalinstruments and other objects and their corresponding virtual geometricvolumes and projected extensions could be determined by the specificorientation of the reflective spheres relative to one another givingeach virtual object an individual identity within the navigation system.This allows the navigation system to identify the medical instrument orother object and its corresponding virtual overlay representation (i.e.the correct overlay volume) as seen as 310 in FIG. 3. The location ofthe markers also provide other useful information to the trackingsystem, such as the medical instrument's central point, the medicalinstrument's central axis and orientation, and other information relatedto the medical instrument. In an embodiment the mentioned usefulinformation may be utilized to define the projected extension of thevolume representing its trajectory such as 320 in FIG. 3. Thistrajectory may be defined arbitrarily given it has medical utility, suchas the trajectory of a suturing needle when used to pierce tissue. Thevirtual overlay representation of the medical instrument may also bedeterminable from a database of medical instruments.

As mentioned above in an embodiment the distance of the distal end ofprobe of the medical instrument 210 from an imminent structure may bedetermined using a laser range finder or a structured light scan. Theseimplementations will be described below in more detail.

Alternative markers may include radio frequency (RF), electromagnetic(EM), pulsed and un-pulsed light emitting diodes (LEDs), glass spheres,reflective stickers, unique structures and patterns. Further, the RF andEM markers may have specific signatures for the specific tools theywould be attached to. The reflective stickers, structures and patterns,glass spheres, LEDs could all be detected using optical detectors, whileRF and EM could be picked up using antennas. Advantages to using EM andRF tags would include removal of the line-of-sight condition during theoperation, whereas using an optical-based tracking system removes theadditional noise and distortion from environmental influences inherentto electrical emission and detection systems.

In a further embodiment, 3-D design markers could be used for detectionby an auxiliary camera and/or optical imaging system. Such markers couldalso be used as a calibration pattern to provide distance information(3D) to the optical detector. These identification markers may includedesigns such as concentric circles with different ring spacing, and/ordifferent types of bar codes. Furthermore, in addition to using markers,the contours of known objects (i.e., side of the port) could be maderecognizable by the optical imaging devices through the tracking system.

For accurate overlays to be produced the first step is to define acommon coordinate space composed of both an actual coordinate space anda virtual coordinate space. Where the actual coordinate space containsactual objects that exist in space, virtual coordinate space containsvirtual objects that are generated in a virtual space, and the commoncoordinate space contains both the aforementioned actual and virtualobjects.

It should be noted that the virtual objects may also be comprised oflandmarks that can be used to associate them (i.e. their spatialpositions and poses) with their respective actual objects. Theselandmarks are placed in predetermined virtual positions relative to thevirtual objects and are correlated with actual landmarks placed inpredetermined positions relative to the respectively associated actualobjects. Examples of such landmarks are provided in FIG. 3. In thefigure virtual landmarks 300 are located in a predetermined positionrelative to the virtual object comprising of the pointer segment 310 andpointer extension 320. Actual landmarks 200 are also located in apredetermined position relative to the actual objects they're connectedto. It should be noted that the virtual and actual objects spatialrelationships (i.e. spatial position and pose) relative to theirrespective virtual and actual landmarks are predefined within thesystem. It should also be noted that the generation of the virtualobject (including its landmarks) in a specified position and pose(spatial relationship) relative to the actual landmarks in the commoncoordinate frame is also predefined within the system. Theserelationships are then used to generate the virtual objects with aspecific position and pose relative to the actual objects position andpose in the common coordinate frame.

An example of such relationships is shown in FIG. 3. In the figure itcan be seen that the virtual landmarks, when aligned with the actuallandmarks create an accurate overlay of the virtual object on the actualobject. In some embodiments the virtual objects may mirror the actualobjects in characteristics such as but not limited to size, shape,texture, colour, location, and etc. For example virtual pointer segment310 mimics the shape and location of the actual objects pointer segmentas depicted in FIG. 3. While in alternate embodiments the virtual objectrepresentation can be an arbitrary size, shape, texture, colour,location, and etc. that provides a useful information to the user. Forexample virtual pointer segment 330 mimics the direction of the pointerhandle and provides information about the location of the tip of theactual pointer, but does not mimic the actual object with respect to itsshape.

In order to form a common coordinate space composed of the amalgamatedvirtual and actual coordinate spaces, the two spaces must be coupledwith a common reference coordinate, having a defined position and posethat can be located in both the actual and virtual coordinate spaces. Anexample of such a reference coordinate 400 and actual and virtualcoordinate space origins, 410 and 420, are provided in FIG. 4. Once thecommon reference coordinate location (i.e. position and pose) isacquired in both spaces they can be used to correlate the position andpose (coordinates) of any point in one coordinate space to the other.The correlation is determined by equating the locations of the commonreference coordinate in both spaces and solving for an unknowntranslation variable for each degree of freedom defined in the twocoordinate spaces. These translation variables may then be used totransform a coordinate in one space to an equivalent coordinate in theother. An example correlation can be derived from the diagram in FIG. 4depicting a two dimensional coordinate space. In the figure the commonreference coordinates 400 position are determined relative to the actualcoordinate space origin 410 and the virtual coordinate space origin 420.These common reference coordinates can be derived from the diagram as:

(X _(cra) ,Y _(cra))=(55,55) and

(X _(crv) ,Y _(crv))=(−25,−45)

Where the subscript “cra” denotes the common reference coordinateposition relative to the actual coordinate space origin and thesubscript “cry” denotes the common reference coordinate positionrelative to the virtual coordinate space origin. Utilizing a generictranslation equation describing any points ((Y_(a), X_(a)) and (Y_(v),X_(v))), where the subscript “a” denotes the coordinates of a pointrelative to the actual coordinate space origin 410, and the subscript“v” denotes the coordinate of a point relative to the virtual coordinatespace origin 420, we can equate the individual coordinates from eachspace to solve for translation variables ((Y_(T), X_(T))), where thesubscript “T” denotes the translation variable as shown below.

Y _(a) =Y _(v) +Y _(T)

X _(a) =X _(v) +X _(T)

Now substituting the derived values of our points from FIG. 4 we cansolve for the translation variable.

55=−45+Y _(T)

100=Y _(T)

and

55=−25+X _(T)

80=X _(T)

Utilizing this translation variable, any point ((i.e. (Y_(v), X_(v))) inthe virtual coordinate space may be transformed into an equivalent pointin the actual coordinate space through the two generic transformationequations provided below. It should be noted that these equations can berearranged to transform any point from the actual coordinate space intoan equivalent point in the virtual coordinate space as well.

Y _(a) =Y _(v)+100

and

X _(a) =X _(v)+80

This will allow the virtual and actual objects respective equivalentpositions and poses to therefore be defined in both the actual andvirtual common coordinate spaces simultaneously. Once the correlation isdetermined the actual and virtual coordinate spaces become coupled andthe resulting common coordinate space can be used to overlay virtual andreal objects when imaged. It should be noted that these virtual and realobjects can be superimposed in the common coordinate space.

Furthermore, the above-mentioned computation can also be used incomputer readable instructions to track the position and orientation (orequivalently pose) of the surgical instrument, the non-contact distanceacquiring device and the position of a proximal surface of an object.Once initial coordinates are generated and stored in the computer bycalibrating the spatial relationship between the non-contact distanceacquiring device and the surgical instrument, using combined readingsfrom the non-contact distance acquiring device, providing the distancebetween the device and the proximal surface of the object, and thereadings from the tracking markers, providing the relative motionbetween the non-contact distance acquiring device and the surgicalinstrument, a computer can track the relative locations of the surgicalinstrument, the non-contact distance acquiring device and object.

The second step in producing an accurate overlay is to identify anactual camera(s) of which the imaging feed is to be overlaid with avirtual imaging feed from a virtual camera, where both cameras arelocated within the common coordinate space. Examples of such cameras areshown in FIG. 5. In the figure a diagram is shown comparing the positionand pose of the virtual and actual cameras, 505 and 515, in the commoncoordinate space and their respective imaging feeds. Once the actualcamera is identified its position and pose in the common coordinatespace 550 must be acquired. When this is complete the virtual cameramust be constrained to have the equivalent spatial position and pose asthe actual camera. This can be accomplished by applying the transformdescribed in detail above. In addition this virtual camera should havethe same optical properties as the actual camera, namely, the samefield-of-view, aspect ratio, and optical distance in order to providethe same perspective view of the common coordinate space as the actualcamera (given the virtual and actual objects have the same location inthe common coordinate space) as depicted in FIG. 5 (C).

The third step in producing an accurate overlay is to project theimaging feed from both the actual and virtual cameras onto a display(allowing for some or no transparency effect in one of the feeds). Thiswill produce an overlay of the virtual objects on their associatedactual object counterparts. The actual camera will capture the view ofthe actual objects in the common coordinate space while the virtualcamera will capture a view of the virtual objects in the commoncoordinate space. FIG. 5 depicts a situation in which the virtual camera505 is not aligned with an actual camera 515 in the common coordinateframe 550 to illustrate the need for a correct alignment of the virtual505 and actual 515 cameras to produce an accurate overlay. In the figurethe display 525 containing imaging feeds from both the actual andvirtual cameras in the common coordinate frame is shown. The arrows 500and 510 represent the discrepancy in alignment of the cameras andconsequently the overlay on the display. As the virtual camera is movedleft from FIG. 5A to FIG. 5B it can be seen that the overlay consistingof 310, 320, and 300 progressively moves right, closer to a correctalignment. As the diagram moves from FIG. 5B through to FIG. 5C alongthe discrepancy path shown by arrow 510, the cameras become coincidentand the overlay moves in the opposite direction in the display to becomecorrectly aligned as can be seen in FIG. 5C. The explanation of FIGS. 5Ato 5C above is to illustrate the effect of incorrect alignment of thecameras and to provide reasoning as to why correct alignment is anecessity when producing accurate overlays. In common practice thevirtual camera and virtual objects will be constantly generatedcoincidently at the location (position and pose) of the actual cameraand corresponding actual objects. In general both cameras will have thesame perspective view of the common coordinate space including anyactual and virtual objects contained within, because of the camerasidentical optical properties and positions and poses. Therefore anyvirtual objects should substantially align with their associated actualobject counterparts, if they are generated in the equivalent positionand pose as them in the common coordinate frame. If this overlay isexecuted periodically this can allow for a real-time overlaid imagingfeed of the surgical site of interest.

The system and method disclosed herein is implemented using at least onenon-contact distance acquiring device in a known relative position andorientation with respect to the surgical instrument. The non-contactdistance acquiring device may be attached to the surgical instrument ormay be at a remote location from the surgical instrument. The systemincludes a computer processor, in data communication with the one ormore non-contact distance acquiring devices. The computer processor isprogrammed with instructions to compute a distance between the surgicalinstrument and the object in the surgical area of interest. Acommunication device for communicating the distance to a user isconnected to the computer processor.

Additionally, a camera for acquiring an image feed of the surgical areaof interest may be included. The camera has a known position andorientation with respect to the surgical instrument, and being ininformation communication with the computer processor, is programmedwith instructions to overlay onto the image feed, generated on a visualdisplay, a visual cue depicting the distance between the surgicalinstrument and the object. The overlay may also depict a projectedtrajectory of the surgical instrument. This projected trajectory maytake the form of a line with specific characteristics. The visual cuemay inform a user of the distance to an object by changing thecharacteristics of the line generated on the visual display at the pointwhere the trajectory would intersect with the object. Some non-limitingexamples of line characteristics that can be changed include color,thickness, line pattern and any combination thereof.

A tracking system can be employed to determine the relative positionsand orientations of surgical equipment located in the operating areaconsisting of one or any combination of the camera, the surgicalinstrument and the distance acquiring device. Using one or more trackingmarkers attachable to the components of the mentioned surgicalequipment, a tracking sensor can continuously monitor relative positionsand orientations of the surgical equipment.

The object in the surgical area of interest can include tissue of apatient being operated on, an implant, or any other objects located inthe surgical area of interest. The distance acquiring device can be alaser range finder, a structured light detection device for 3D imaging,an ultrasonic transducer, or any other non-contact device capable ofdetermining the distance of an object relative to itself.

The detector may include an MRI, a CT scanner, an X-ray scanner, a PETscanner, an ultrasonic scanner or a digital camera. Non-limitingexamples of the visual display may be a digital display, a heads-updisplay, a monitor, a navigation instrument display or a microscopedisplay.

Distance thresholds can be stored in the computer such that when thedistance between the surgical instrument and the object reduces below orreaches the threshold distance the computer processor is programmed withinstruction to signal an alert. The alert can be a visual alertgenerated on the visual display or elsewhere, an audio alert or atactile alert such as activating a vibrating member.

An exemplary embodiment of the system disclosed herein is implementedusing a laser range finder as shown in FIG. 6. The laser range finder600 may be mounted to the distal end of a tool 210 for example asdepicted in FIG. 6A. As the tool 210 is moved along the path 620 thelaser range finder 600 functions by emitting a laser pulse 610 andawaiting the return of the reflected laser pulse 625 reflected off ofthe object 630 in the direction of the initial pulse 610 as depicted inFIG. 6B. Once the return pulse 625 is detected the laser range finder600 then calculates the distance to the object 630 that reflected thelaser pulse 625. The distance may then be used to alter an overlay, inparticular the projected extension 320 at the distance where an imminentobject is located providing the surgeon with a visual cue. This can beseen in FIG. 6C where the virtual imaging feed from a virtual camera 515is shown on the display 525 independent of the actual camera imagingfeed for the shown perspective in this view. It should be noted that inthis view of the common coordinate space only virtual objects arevisible in the absence of the actual camera imaging feed. In FIG. 6C thevirtual object representation of the tool 310, in particular itsprojected extension 320 converts from a dotted extension (indicative ofno object) to a dashed extension 660 (indicative of an imminent object)at the point 670 where the imminent object 630 is detected by the laserrange finder 600.

The full effect of the virtual projected extension can be seen in FIG.6D, where the imaging feed from the actual camera 505 and the virtualcamera 515 (FIG. 5) are overlaid on the display 525. In the figure thedisplay 525 (FIG. 6D) contains both the real and virtual imaging feeds,and it can be seen that the projected extension 320 can aid a user indetermining the distance of an object 630 from the distal end of asurgical tool 210. This is valuable information during navigated surgeryas when the surgeon is using a two dimensional display such as thedisplay 525 shown in FIG. 6D there is no depth information available onit and the surgeon's own depth perception is rendered inutile. Thereforeindicating the depth of an imminent object in the direction of a toolstrajectory to a surgeon, especially when working in a small corridortype setup for a surgery such as the port based surgery mentioned above,allows the surgeon to perform the procedure with greater accuracy andpotentially in less time.

In an alternate embodiment the distance of the distal end of the toolfrom an imminent structure, in the context of an overlaid image of thesurgical field of interest, may be determined using structured lightbased surface scanning. Referring to FIG. 7A it can be seen that in theleft frame there is an imminent object 700 in the trajectory 750 of thesurgical instrument. The right frame contains a structured light surfacerendering device 710 consisting of two light sensors and a projector.The exemplary structured light device 710 functions by projecting aknown structured light pattern 730 onto an object to be rendered,imaging the resultant structured light pattern on the object 740, andthen comparing the known structured light pattern with the imaged one toinfer the 3D surface of the object.

Once the 3D structure has been inferred it can be transferred into thecommon coordinate frame as a virtual object in order to interact withother virtual objects, such as the virtual object representation of thetool 210 consisting of landmarks 300, pointer segment 310, and projectedextension 320. This can be achieved through the use of a tracking deviceas described above, an example of which is shown as 768 in FIG. 7A.Using the tracking device the location (spatial position and pose) ofthe structured light surface rendering device 710 can be determined inthe common coordinate frame if the structured light surface renderingdevice has tracking markers, such as tracking markers 765 shown in FIG.7A.

Once the location of the of the surface rendering device 710 is known inthe common coordinate space, the 3D surface of the object 700 that wasdetected by the device 710 can be rendered as a virtual surface in thecommon coordinate space (because its location becomes known relative tothe structured light device) as shown in the left frame of FIG. 7B. Theright frame in FIG. 7B depicts the imaging feed of a virtual camera 515in the display 525. The scene being captured by the imaging feed depictsthe interaction of all of the generated virtual objects in the commoncoordinate frame captured by the virtual camera 515.

Given the virtual object representation 780 of the actual object 700 isnow represented in the common coordinate frame. The projected extension320 in this frame can be configured to change from a dotted extension320 (indicative of no object) into a dashed extension 660 (indicative ofan imminent object) at the point whenever the virtual projectedextension comes into contact with the virtual object representation 780,such as the point 670 shown in FIG. 7B. The full effect of the virtualprojected extension can be seen in the right frame of FIG. 7B, where theimaging feed from the actual camera 505 and the virtual camera 515 areoverlaid on the display 525. In the figure the display 525 contains boththe real and virtual imaging feeds, and it can be seen that theprojected extension 320 can aid a user in determining the distance of anobject 700 from the distal end of a surgical tool 210. This is valuableinformation during navigated surgery as when the surgeon is using a twodimensional display, such as the display 525 shown in FIG. 7, there isno depth information available on it and the use of a screen renders thesurgeon's own depth perception inutile. Therefore indicating the depthof an imminent object in the direction of a tool's trajectory to asurgeon, especially when working in a small corridor type setup for asurgery such as the port based surgery mentioned above, allows thesurgeon to perform the procedure with greater accuracy. An example of amock brain surgery procedure is depicted in FIG. 8 where the projectedtrajectory of the tool can be seen to change a characteristic as itpenetrates the brain volume.

In addition to the mentioned embodiments it may also serve as useful toindicate on the display the distance of the projected trajectory beforeit contacts an imminent structure as shown as 800 in FIG. 8.

The specific embodiments described above have been shown by way ofexample, and it should be understood that these embodiments may besusceptible to various modifications and alternative forms. It should befurther understood that the claims are not intended to be limited to theparticular forms disclosed, but rather to cover all modifications,equivalents, and alternatives falling within the spirit and scope ofthis disclosure.

1-34. (canceled)
 35. A method for communicating a distance of a surgicalinstrument from an object in a surgical area of interest, comprising:determining a relative position and orientation between at least onenon-contact distance acquiring device, having a known relative positionand orientation, and a the surgical instrument, thereby providing adetermined relative position and orientation between the at least onenon-contact distance acquiring device and the surgical instrument;acquiring a first distance, between said at least one non-contactdistance acquiring device, having the known relative position andorientation, and the object in the surgical area of interest; computing,using the determined relative position and orientation between the atleast one non-contact distance acquiring device and the surgicalinstrument and the first distance, a second distance between thesurgical instrument and the object; communicating the second distance;and overlaying onto the image feed a projected trajectory of thesurgical instrument.
 36. The method according to claim 35, wherein thevisual cue is provided by changing one or more line characteristics ofthe projected trajectory.
 37. The method according to claim 35, whereinsaid one or more line characteristics include a color, thickness, linepattern, and any combination thereof.
 38. The method according to claim35, wherein the known relative position and orientation of one or morenon-contact acquiring devices and the imaging detector with respect tothe surgical instrument is acquired by a tracking system.
 39. The methodaccording to claim 35, wherein the tracking system includes one or moretracking markers on at least the surgical instrument and a trackingsensor for tracking said one or more tracking markers.
 40. The methodaccording to claim 35, wherein the object in a surgical area of interestis tissue of a patient being operated on.
 41. The method according toclaim 35, wherein the non-contact distance acquiring device is a laserrange finder.
 42. The method according to claim 35, wherein thenon-contact distance acquiring device is a structured light detectiondevice for 3D imaging.
 43. The method according to claim 35, wherein thenon-contact distance acquiring device is an ultrasonic transducer. 44.The method according to claim 35, wherein the non-contact distanceacquiring device is attached to the surgical instrument.
 45. The methodaccording to claim 35, wherein the non-contact distance acquiring deviceis at a remote location from the surgical instrument.
 46. The methodaccording to claim 35, wherein the imaging detector is selected from agroup consisting of a digital camera, a MRI, a CT scanner, an X-rayscanner, a PET scanner, or an ultrasonic scanner.
 47. The methodaccording to claim 35, wherein the visual display is a digital display,a heads-up display, a monitor, a navigation instrument display or amicroscope display.
 48. The method according to claim 35, wherein atleast one distance threshold is programmable into the computer such thatwhen the distance between the surgical instrument and the object reducesbelow or reaches the at least one distance threshold, an alert istriggered.
 49. The method according to claim 48, wherein the alert is avisual alert, an audio alert or a tactile alert.
 50. The methodaccording to claim 35 further comprising displaying the depth of theobject in the surgical area of interest in the direction of thetrajectory of the surgical instrument.